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Transmitting Recorded Material 

The present invention is concerned with methods and apparatus for transmitting 
recorded material, such as video, audio of other material to be played in real time, over 
5 a network. 

According to one aspect of the present invention there is provided a method of 
transmitting a recording comprising: 

- commencing transmission thereof; 

- holding received data in a receiver buffer; and 
10 - commencing playing of said received data; 

characterised by the steps of analysing the whole of the recording to determine a 
point at which to commence playing such that no buffer underflow can occur; and 
commencing playing only when this point has been reached. 

In another aspect, the invention provides a method of transmitting a recording 
15 comprising: 

- commencing transmission thereof; 

- holding received data in a receiver buffer; and 

- commencing playing of said received data; 
characterised by the steps of: 

20 analysing the whole of the recording to identify a first section at the beginning thereof 
which meets the condition that it covers a playing time interval greater than or equal to 
the maximum of the timing error for a following section of any length, each timing error 
being defined as the extent to which the transmission time of the respective following 
section exceeds its playing fime interval; and causing the receiver to commencing 

25 playing only after said first section has been received. 

Further aspects of the invention are set out in the claims 

Some embodiments of the invention will now be described, by way of example, with 
reference to the accompanying drawings, in which: 

Figure 1 is a block diagram of a transmission system embodying the invention; 
30 Figure 2 is a timing di^^gram; ::r - 

•" . - •- •> 

Figure 3 is a flaWchart explaining the operation of the control unit shown in Figure 1 ; 





Figure 4 is a flowchart explaining an alternative mode of operation of the control unit; 
and 

Figure 5 is a flowchart explaining a yet further version. 

In Figure 1 , a streamer 1 contains (or has access to) a store 1 1 in which are stored files 
5 each being a compressed version of a video sequence, encoded using a conventional 
compression algorithm such as that defined in the ITU standard H.261 or H.263, or one 
of the ISO MPEG standards. Naturally one may store similar recordings of further video 
sequences, but this is not important to the principles of operation. 

By "bit-rate" here is meant the bit-rate generated by the original encoder and consumed 
10 by the ultimate decoder; in general this is not the same as the rate at which the 
streamer actually transmits, which will be referred to as the transmitting bit-rate. It 
should also be noted that these files are generated at a variable bit-rate (VBR) - that is, 
the number of bits generated for any particular frame of the video depends on the 
picture content. Consequently, references above to low (etc.) bit-rate refer to the 
15 average bit-rate. 

The server has a transmitter 12 which serves to output data via a network 2 to a 
terminal 3. The transmitter is conventional, perhaps operating with a well known 
protocol such as TCP/IP. A control unit 13 serves in conventional manner to receive 
requests from the terminal for delivery of a particular sequence, and to read packets of 

20 data from the store 1 1 for sending to the transmitter 12 as and when the transmitter is 
able to receive them. Here it is assumed that the data are read out as discrete 
packets, often one packet per frame of video, though the possibility of generating more 
than one packet for a single frame is not excluded. (Whilst is in principle possible for a 
single packet to contain data for more than one frame, this is not usually of much 

25 interest in practice). 

Note that these packets are not necessarily related to any packet structure used on the 
network 2. 

The terminal 3 has a receiver 31 , a buffer 32, and a decoder 33. 

Some networks (including TCP/IP networks) have the characteristic that the available 
30 transmitting data rate fluctuates according to the degree of loading on the network. 

Some theoretical discussion is in order at this point 




As shown in Figure 2, an encoded video sequence consists of N packets. Each packet 
has a header containing a time index tj (i=0 ..: N-1) (in ternns of real display time - e.g. 

this could be the video frame number) and contains bj bits. This analysis assumes that 
packet / must be completely received before it can be decoded (i.e. one must buffer the 
5 whole packet first). 

In a simple case, each packet corresponds to one frame, and the time-stamps tj 
increase monotonically, that is, > ti for all i. If however a frame can give rise to two 
or more packets (each with the same tj) then >. . If frames can run out of capture- 
and-display sequence (as in MPEG) then the /, do not increase monotonically. Also, in 
10 practice, some frames may be dropped, so that there will be no frame for a particular 
value of . 

These times are relative. Suppose the receiver has received packet 0 and starts 
decoding packet 0 at time tre/^ to. At "time now" of tref + tg the receiver has received 
packet tg (and possibly more packets too) and has just started to decode packet g, 

15 Packets giohA are in the buffer. Note that (in the simple case) if ^ - g + 1 then the 
buffer contains packet g only. At time tre/"^ ^ the decoder is required to start decoding 
packet j\ Therefore, at that time v + tj the decoder will need to have received all 
packets up to and including packet y. 

The time available from now up to tre/^ tj is tj) - (tre/ + = - ^g. (1) 



20 The data to be sent in that time are that for packets h toy, viz. 

which at a transmitting rate R will require a transmission duration 



(2) 



(3) 



R 

This is possible only if this transmission duration is less than or equal to the time 
25 available, i.e. when the currently available transmitting rate R satisfies the inequality 
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— — <t -t^ 



(4) 



Note that this is the condition for satisfactory reception and decoding of packet 
satisfactory transmission of the whole of the remaining sequence requires that this 
condition be satisfied for all j = h ... NA. 

5 For reasons that will become apparent, we rewrite Equation (4) as: 



J 

R 



J J 

Note that tj - tf,_i = X i^i - h-\ ) = Z "^^^^^ = U ~ - 
Also, we define A^^. = (b^ I R)- At^ 

Note that t^-i'^^g difference between the time-stamp of the most recently 

10 received packet in the buffer and the time stamp of the least recently received packet in 
the buffer - i.e. the one that we have just started to decode. 

Then the condition is 

t,Ae,<t,_,,-t^ (6) 

For a successful transmission up to the last packet N-l, this condition must be satisfied 

15 for any possible j\ viz. 



The left-hand side of Equation (7) represents the maximum timing error that may occur 
from the transmission of packet h up to the end of the sequence, and the condition 
states, in effect that this error must not exceed the ability of the receiver buffer to 
20 accommodate it, given Its current contents. For convenience, we will label the left- 
hand side of Equation (7) as Th - i.e. 



(8) 
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So that Equation (7) may be written as 

T,^h^^-t, (9) 

Consider the situation at time tg = to, that is, when the decoder is to commence 
decoding of the first packet. In the general case, the above condition will not be 
5 satisfied when there is only one packet in the buffer (/i=1). The receiver waits for the 
buffer contents to reach a satisfactory level before it commenced decoding. Using the 
above condition, it becomes apparent that the receiver should wait at least until the 
buffer contains packet H-^ where H is the smallest value of h for which the condition 

TH^h_,-t, (10) 
10 is satisfied. 

In this embodiment of the invention, one of the functions of the control unit 13 is that, 
each time it sends a packet to the transmitter 12, it evaluates the test embodied in 
Equation 10. 

Figure 3 is a flowchart showing operation of the control unit. At step 101 a packet 
15 counter is reset. Then (102) the first packet (or on subsequent iterations, the next 
packet) is read from the store 11 and sent to the transmitter 12. At step 103, the 
control unit computes the value of T„. At this point, the counter n points to the last 
packet sent, whereas Equation (10) is formulated for the last packet sent being /i-1. 
Consequently the calculation at step 103 is of Tn^^ and the test performed at step 104 

20 is whether T^^^ <t„'-tQ. 

If the test is passed, then it is known that the receiver is safe to begin decoding as soon 
as it has received this packet. Therefore at step 105 the control unit sends to the 
transmitter a "start" message to be sent to the receiver. When the receiver receives 
this start message, it begins decoding. If there is any possibility of messages being 
25 received in a different order from that in which they were sent, then the start message 
should contain the packet index n so that the receiver may check that packet n has 
actually been received before it commences decoding. Alternatively, the transmitter 
could send values of r„+^ to the receiver, and the receiver itself performs the test. 

The packet counter is incremented at 106 and control returns to step 102 where, as 
30 soon as the transmitter is ready to accept it, a further packet is read out and 
transmitted. 
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The preceding description assumes that the control unit performs this calculation each 
time it sends a packet to the transmitter, which is computationally quite intensive. An 
alternative is to perform the calculation less often, perhaps once every five packets, 
5 which reduces the amount of computation but may result in the buffering of more 
frames than is necessary. 

Another alternative is to complete the computation as soon as it is able to do so (i.e. 
without waiting for the next packet) and then send a start message (with starting packet 
number) to the receiver. A yet further alternative is to perform the computation before 

10 transmitting any packets at all. Once the value of h is determined, we then transmit 
packets 0 to A-1 in reverse order (packet h -1 , packet /i - 2, ... packet 0). In this case it 
ceases to be necessary to transmit an explicit "start" command. Standard receivers 
that support UDP transport protocol are able to reorder packets, and will automatically 
wait until packet 0 has arrived before commencing decoding. In fact, it is sufficient that 

15 packet 0 is withheld until after packets 1 to ^ -1 have been sent (whose order is 
immaterial). 

This however precludes the possibility of taking into account changes in the 
transmitting data rate R during the waiting period, and is therefore satisfactory only if 
such changes are not expected. 

20 Observe (by inspection of Equation (3)) that the significance of the rate R is in 
calculating the time taken to send packets h to j. Therefore the actual rate used to 
transmit packets 0 to ^ -1 is of no consequence as it does not affect the result. 

Another attractive option is to perform as much as possible of the computation in 
advance. If a system in which only one value of R is possible, or permitted, then the 

25 computation of r«^i at step 103 and the test of step 104 can be performed in advance 
for each frame up to the point where the test is passed, and the result recorded in the 
file, for example by recording the corresponding value of w in a separate field at the 
start of the file, or by attaching a special flag to frame n itself. Thus in Figure 3, steps 
103 and 104 would be replaced by the test "is current value of n equal to the value of n 

30 stored in the file?"; or "does current frame contain the start flag?". Alternatively the 
separate field (or flag) could be forwarded to the receiver and this recognition process 
performed at the receiving end. 
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Figure 4 shows a flowchart of a process for dealing with the situation where the 
transmitting data rate R varies. In principle this involves 7/, for every packet and storing 
this value in the packet header. In practice however It is necessary to compute them 
for a sufficient number of frames (perhaps 250 frames at 25 frames per second) at the 
5 beginning of the sequence that one is confident that the test will be passed within this 
period. Unfortunately, the calculation of Th involves the value of R, which is of course 
unknown at the time of this pre-processing. Therefore we proceed by calculating 7/, for 
a selection of possible values of R. for example (if Ra is the average bit rate of the file 
in question) 

10 R^=R^ 

i?4 = 1 3R ^ 

So each packet h has these five precalculated values of 7/, stored in it. If required (for 
the purposes to be discussed below) one may also store the relative time position at 
which the maximum in Equation (8)) occurs, that is, 

^hmsK = Omax "^A "^^^^^^jmsoi ^^e value of j in Equation 8 for which Th is obtained. 
15 In this case the flowchart proceeds as follows following transmission of frame n: 
107: interrogate the transmitter 12 to determine the available transmitting rate R] 

103A:EITHER - in the event that R corresponds to one of the rates for which Th has 
been precalculated - read this value from the store; 

OR - in the event that R does not so correspond, read from the store the value of Th 
20 (and, if required, tkmax) that correspond to the highest one {R ') of the rates R1...R5 that is 
less than the actual value of i?, and estimate Tf, from it; 

1 04A: Apply the test 7;+, + A < r„ - ^0 . where ^ is a fixed safety margin; 
Continue as before. 

The estimate of could be performed simply by using the value Th associated with R "; 
25 this would work, but sincis it would overestimate 7), it would result, at times, in the 
receiver waiting longer than necessary. Another option would be by linear (or other) 
interpolation between the values of Th stored for the two values of /?i ... R5 each side of 



• 
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the actual value R, However, our preferred approach is to calculate an estimate 
according to: 

j^^ jTr+At-.r.^W (11) 

R 

Where R ' is the highest one of the rates Ri ... Rs that is less than the actual value of R, 
5 Tf is the precalculated n for this rate, At;^ is the time from ti at which T," is obtained 
(i.e. is the accompanying value of A^^^a^)- the event that this method returns a 
negative value, we set it to zero. 

Note that this is only an estimate, as is a nonlinear function of rate. However with 
this method 2} ' is always higher than the true value and automatically provides a safety 
1 0 margin (so that the margin A shown above may be omitted). 

Note that these equations are valid for the situation where the encoding process 
generates two or more packets (with equal t,) for one frame, and for the situation 
encountered in MPEG with bidirectional prediction where the frames are transmitted in 
the order in which they need to be decoded, rather than in order of ascending 

15 We will now describe an alternative embodiment in which the mathematics is converted 
Into an equivalent form which however, rather than performing the calculations for each 
pacl<et individually, makes use of calculations already made for a preceding packet. 
Recalling Equation(8): 

20 which may be rewritten 

n=MaX \iAs„McXX~~'''{i^e, + iAs, W (12) 



-Max [^£h> Max^^^^^ |a^. + 2^a^, 



= ls.€f^ +Max jp, Tj 



(13) 



Provided that T^^, > 0 , which will be true at the beginning of the file, this becomes 



(14) 



Or generally 



a+b-\ 



i=a 



If b=h-a then 



substituting Asj = ~ A/'; and At^ = ^/ - ^/-i 

R 



(15) 



A i4 44 A 



(16) 



If a=0, then 



If a=1 , then 



Consider the test 



which may be written 



(17) 
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if a=0, this becomes 

^o-S l^.+^-.-^o (18) 

Noting that ti is a meaningless quantity (appearing on both sides on the inequality) so 
that it can be given any value, it is convenient to define ti as equal to to, whence we 
5 obtain 

(or, if a = 1: 7]<2 ^) 
Thus the test of Equation (1 0) 

10 could instead be written 

/=0 ^ 

Then the first test (h=1) is Test 1: 
° R 



Or, if we define = 7^ -— > ^he first test is Zj < 0 ? 



1 5 The second test is Z^^O 



The xth test is Z^ :< 0 



1 ^ \ ^ b b 

But = To -^S^/ = ^0 ~ = Z^- 



R i^o ' R 1=0 ^ ^ R 

So each test can update the previous value of Z, as shown in the flowchart of Figure 5. 
First, at Step 201, To is calculated in accordance with Equation (8), then (Step 202) Zq 
20 is set equal to To- At step 203 a packet counter is reset. Then (204) the first packet (or 
on subsequent iterations, the next packet) is read from the store 1 1 and sent to the 
transmitter 12. At step 205, the control unit computes the value of Z^+t, and the test 



performed at step 206 as to whether Z„+, < 0 . If the test is passed, then it is l<nown 

that the receiver is safe to begin decoding as soon as it has received this packet 
Therefore at step 207 the control unit sends to the transmitter a "start" message to be 
sent to the receiver. When the receiver receives this start message, it begins 
decoding. The pacl<et counter is incremented at 208 and control returns to step 204 

where, as soon as the transmitter is ready to accept it, a further packet is read out and 

It 

transmitted. 

The step 201 of calculating To could be done in advance and the values stored. This 
procedure could of course be adapted, in a similar manner to that previously described, 
to accommodate different values of /2. 

It is not essential that this process begin with Tq. One could start with (in which case 
the first test is T, < 0 ?) or, if one chooses always to buffer at least two (or more) 
packets one could start with Ta, etc. 

Although the example given is for encoded video, the same method can be applied to 
encoded audio or indeed any other material that is to be played in real time. 

If desired, in multiple-rate systems, these methods may be used in combination with 
the rate-switching method described in our international patent application dated 23"*^ 
March 2004, which claims priority from U.K. patent application 0306973.9 of 26**" March 
2003. 




CLAIMS 

1 . A method of transmitting a recording comprising: 

- commencing transmission thereof; 

- holding received data in a receiver buffer; and 
5 - commencing playing of said received data; 

characterised by the steps of analysing the whole of the recording to determine a point 
at which to commence playing such that no buffer underflow can occur; and 
commencing playing only when this point has been reached. 

10 2. A method of transmitting a recording comprising: 

- commencing transmission thereof; 

- holding received data in a receiver buffer; and 

- commencing playing of said received data; 
characterised by the steps of: 

1 5 analysing the whole of the recording to identify a first section at the beginning thereof 
which meets the condition that it covers a playing time interval greater than or equal to 
the maximum of the timing error for a following section of any length, each timing error 
being defined as the extent to which the transmission time of the respective following 
section exceeds its playing time interval; and causing the receiver to commencing 

20 playing only after said first section has been received. 

3. A method according to claim 2 comprising, after transmission of said first portion, 
transmitting an instruction to the receiver to commence playing. 

25 4, A method according to claim 2 comprising transmitting to the receiver an instruction 
specifying the first section and wherein the receiver commences playing when it 
recognises that the first section is in the buffer. 

5. A method according to claim 2 in which the analysis comprises: 
30 (a) at the transmitter, computing said maximum timing error values for different portions 
of the sequence, and 




(b) at the receiver, comparing the values with the buffer contents to recognise when 
said first section is in the buffer. 

6. A method according to claim 2 comprising withholding transmission of an initial part 
5 of the recording until the remainder of said first section has been transmitted; 

transmitting said initial part; and wherein the receiver commences playing only when 
said initial part is received. 

7. A method according to claim 2,3, 4 or 6 including performing the analysis in advance 
10 and marking the identified section in the recording. 

8. A method according to any one of claims 2 to 6 where said analysis includes 
computing, in advance, timing error values corresponding to a plurality of transmitting 
data rates and storing them; and subsequently estimating therefrom an error value 

15 corresponding to an actual transmitting data rate. 

9. A method according to any one of the preceding claims in which the analysis 
comprises testing a timing error parameter evaluated for successive portions of the 
recording, wherein the timing error parameter is firstly calculated in respect of a first or 

20 early portion of the recording and the timing error parameter for subsequent portions is 
obtained by updating the parameter obtained for the preceding portion. 

10. A method according to any one of the preceding claims in which the recording is a 
video recording. 

25 

1 1 . A method according to any one of the preceding claims in which the recording is an 
audio recording. 
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